Learning Single-Image Depth from Videos using Quality Assessment Networks

Depth estimation from a single image in the wild remains a challenging problem. One main obstacle is the lack of high-quality training data for images in the wild. In this paper we propose a method to automatically generate such data through Structure-from-Motion (SfM) on Internet videos. The core of this method is a Quality Assessment Network that identifies high-quality reconstructions obtained from SfM. Using this method, we collect single-view depth training data from a large number of YouTube videos and construct a new dataset called YouTube3D. Experiments show that YouTube3D is useful in training depth estimation networks and advances the state of the art of single-view depth estimation in the wild

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation

The images and sounds that we perceive undergo subtle but geometrically consistent changes as we rotate our heads. In this paper, we use these cues to solve a problem we call Sound Localization from Motion (SLfM): jointly estimating camera rotation and localizing sound sources. We learn to solve these tasks solely through self-supervision. A visual model predicts camera rotation from a pair of images, while an audio model predicts the direction of sound sources from binaural sounds. We train these models to generate predictions that agree with one another. At test time, the models can be deployed independently. To obtain a feature representation that is well-suited to solving this challenging problem, we also propose a method for learning an audio-visual representation through cross-view binauralization: estimating binaural sound from one view, given images and sound from another. Our model can successfully estimate accurate rotations on both real and synthetic scenes, and localize sound sources with accuracy competitive with state-of-the-art self-supervised approaches. Project site: https://ificl.github.io/SLfM/Comment: ICCV 2023. Project site: https://ificl.github.io/SLfM

OASIS: A Large-Scale Dataset for Single Image 3D in the Wild

Single-view 3D is the task of recovering 3D properties such as depth and surface normals from a single image. We hypothesize that a major obstacle to single-image 3D is data. We address this issue by presenting Open Annotations of Single Image Surfaces (OASIS), a dataset for single-image 3D in the wild consisting of annotations of detailed 3D geometry for 140,000 images. We train and evaluate leading models on a variety of single-image 3D tasks. We expect OASIS to be a useful resource for 3D vision research. Project site: https://pvl.cs.princeton.edu/OASIS.Comment: Accepted to CVPR 202

Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practices

While Graph Neural Networks (GNNs) are remarkably successful in a variety of high-impact applications, we demonstrate that, in link prediction, the common practices of including the edges being predicted in the graph at training and/or test have outsized impact on the performance of low-degree nodes. We theoretically and empirically investigate how these practices impact node-level performance across different degrees. Specifically, we explore three issues that arise: (I1) overfitting; (I2) distribution shift; and (I3) implicit test leakage. The former two issues lead to poor generalizability to the test data, while the latter leads to overestimation of the model's performance and directly impacts the deployment of GNNs. To address these issues in a systematic way, we introduce an effective and efficient GNN training framework, SpotTarget, which leverages our insight on low-degree nodes: (1) at training time, it excludes a (training) edge to be predicted if it is incident to at least one low-degree node; and (2) at test time, it excludes all test edges to be predicted (thus, mimicking real scenarios of using GNNs, where the test data is not included in the graph). SpotTarget helps researchers and practitioners adhere to best practices for learning from graph data, which are frequently overlooked even by the most widely-used frameworks. Our experiments on various real-world datasets show that SpotTarget makes GNNs up to 15x more accurate in sparse graphs, and significantly improves their performance for low-degree nodes in dense graphs.Comment: Extended Version of our WSDM'24 paper. 8 pages, 2 page appendi

A hub gene signature as a therapeutic target and biomarker for sepsis and geriatric sepsis-induced ARDS concomitant with COVID-19 infection

BackgroundCOVID-19 and sepsis represent formidable public health challenges, characterized by incompletely elucidated molecular mechanisms. Elucidating the interplay between COVID-19 and sepsis, particularly in geriatric patients suffering from sepsis-induced acute respiratory distress syndrome (ARDS), is of paramount importance for identifying potential therapeutic interventions to mitigate hospitalization and mortality risks.MethodsWe employed bioinformatics and systems biology approaches to identify hub genes, shared pathways, molecular biomarkers, and candidate therapeutics for managing sepsis and sepsis-induced ARDS in the context of COVID-19 infection, as well as co-existing or sequentially occurring infections. We corroborated these hub genes utilizing murine sepsis-ARDS models and blood samples derived from geriatric patients afflicted by sepsis-induced ARDS.ResultsOur investigation revealed 189 differentially expressed genes (DEGs) shared among COVID-19 and sepsis datasets. We constructed a protein-protein interaction network, unearthing pivotal hub genes and modules. Notably, nine hub genes displayed significant alterations and correlations with critical inflammatory mediators of pulmonary injury in murine septic lungs. Simultaneously, 12 displayed significant changes and correlations with a neutrophil-recruiting chemokine in geriatric patients with sepsis-induced ARDS. Of these, six hub genes (CD247, CD2, CD40LG, KLRB1, LCN2, RETN) showed significant alterations across COVID-19, sepsis, and geriatric sepsis-induced ARDS. Our single-cell RNA sequencing analysis of hub genes across diverse immune cell types furnished insights into disease pathogenesis. Functional analysis underscored the interconnection between sepsis/sepsis-ARDS and COVID-19, enabling us to pinpoint potential therapeutic targets, transcription factor-gene interactions, DEG-microRNA co-regulatory networks, and prospective drug and chemical compound interactions involving hub genes.ConclusionOur investigation offers potential therapeutic targets/biomarkers, sheds light on the immune response in geriatric patients with sepsis-induced ARDS, emphasizes the association between sepsis/sepsis-ARDS and COVID-19, and proposes prospective alternative pathways for targeted therapeutic interventions

Understanding 3D Object Interaction from a Single Image

Humans can easily understand a single image as depicting multiple potential objects permitting interaction. We use this skill to plan our interactions with the world and accelerate understanding new objects without engaging in interaction. In this paper, we would like to endow machines with the similar ability, so that intelligent agents can better explore the 3D scene or manipulate objects. Our approach is a transformer-based model that predicts the 3D location, physical properties and affordance of objects. To power this model, we collect a dataset with Internet videos, egocentric videos and indoor images to train and validate our approach. Our model yields strong performance on our data, and generalizes well to robotics data

Novel Drug Delivery Systems: An Important Direction for Drug Innovation Research and Development

The escalating demand for enhanced therapeutic efficacy and reduced adverse effects in the pharmaceutical domain has catalyzed a new frontier of innovation and research in the field of pharmacy: novel drug delivery systems. These systems are designed to address the limitations of conventional drug administration, such as abbreviated half-life, inadequate targeting, low solubility, and bioavailability. As the disciplines of pharmacy, materials science, and biomedicine continue to advance and converge, the development of efficient and safe drug delivery systems, including biopharmaceutical formulations, has garnered significant attention both domestically and internationally. This article presents an overview of the latest advancements in drug delivery systems, categorized into four primary areas: carrier-based and coupling-based targeted drug delivery systems, intelligent drug delivery systems, and drug delivery devices, based on their main objectives and methodologies. Additionally, it critically analyzes the technological bottlenecks, current research challenges, and future trends in the application of novel drug delivery systems

Understanding Light Harvesting in Radial Junction Amorphous Silicon Thin Film Solar Cells

International audienc

Boosting light emission from Si-based thin film over Si and SiO_2 nanowires architecture

International audienc

Bi-Sn alloy catalyst for simultaneous morphology and doping control of silicon nanowires in radial junction solar cells

International audienc